The Magento Config: Loading System Variables

This article is part of a longer series exploring the Magento global configuration object. While this article contains useful stand-alone information, you’ll want to read part 1 and part 2 of the series to fully understand what’s going on.

When our last article finished up, we had successfully loaded a list of every declared module, as well as each of those module’s config.xml files into the main global configuration object’s root node. It seemed like our job was done. However, there’s one last step that need to be taken, and before we can talk about that, we need to talk about Magento’s system configuration variables

System Configuration

If you’ve spent, or are going to spend, a significant time around Magento, you need to get used to a certain amount of flexibility around definitions. So far this series of articles has talked about “Magento Configuration”, where we’re defining Magento Configuration as the config.xml files added to each individual module. Things get tricky when you consider the other types of configuration that are available and necessary in Magento.

For example, all those layout XML files are often considered configuration, but they have almost nothing to do with the global configuration we’ve been talking about. So both systems are configuration, but they’re still two separate systems.

Another such system is the one I’ve been calling Magento’s system configuration variables. If you’re familiar with the system you know it’s a way to quickly configure a user interface for entering values that a Magento module developer may want a Magento user to change.

Once setup, a Magento module developer can make a call like this

to fetch one of those configuration values. This is most often used to create on/off settings for features, store changeable strings, etc. If you’ve dug into this system you know after saving a value via the Magento admin interface, it’s persisted to the core_config_data database table.

You may also know, in the abstract, that directly changing these values in the database table isn’t enough to change a configuration value. You also need to clear Magento’s cache.

Another seemingly weird bit is the default values are not stored in the database, or in the system.xml files used to configure the interface. Instead, module developers add a <default/> node to the global configuration tree via a module’s config.xml file. Values stored here become the default. The core_config_data table only stores values that are explicitly set, along with the scope they’re set at (default, websites, or store).

To a newcomer this system seems cumbersome, hard to understand, and hard to explain to others. That’s because there’s one key assumption that’s not immediately obvious.

That assumption is this: The ultimate destination and “source of truth” for configuration values is the global configuration tree. The core_config_data table is simply a persistence way-station for these user set, scoped values. When you’re reading out a value with getStoreConfig, Magento is looking at the global configuration object.

Starting with Default

With that in mind, let’s pretend we’re developing a configuration system from scratch. Assuming a bare, un-scoped system of key/value lookups, we might say something like

Alright, let’s create a top level node in our configuration tree, and store values there

This node is the <default/> node. At this point, we’re rather pleased with ourselves, and start storing all sorts of useful configuration values. After working with such a system for a bit, we quickly discover a pattern develops where specific websites and/or stores need mostly the same configuration values, with one or two exceptions. Rather than duplicate every set of configuration values, we decide a scoped configuration system is in order.

That is, we want to be able to say all nodes at foo/baz/* have a particular value, except for foo/baz/barwhich has a value of XXXX in this store, YYYY in this website, and ZZZZ everywhere else. Since we’ve already decided all configuration values should live in the global configuration tree, we quickly add two new top level nodes

to store values with the scope of <websites/> and <stores/>. Once again, we’re very pleased with ourselves, and continue coding away on our store.

While this purely XML based configuration system works great for us, (the developers), non-technical end-users are having all sorts of trouble. It’s decided a user interface is needed to allow users to set their own custom values. As developers, we’re game to create this system, but the idea of a system that’s writing directly to config.xml files sets off all sorts of red flags. A simple persistence bug could destroy the entire module configuration, and a clever exploit of the bug could give malicious users the ability to rewrite the configuration of modules however they saw fit.

So, because of this, our system configuration editing system will work like this

1. Users will be able to set values for any configuration node at any scope, but these values will be persisted elsewhere (the core_config_data table)
2. When loading the global config, we’ll look at this elsewhere (core_config_data) and load any new values into the global config

From an outside point of view, it’s easy to scoff at the complexity involved, but if you consider the system from the Varien/Magento point of view, it’s easier to understand how it came to be. Modern agile software development practices rarely leave time to consider the system as a whole; modern systems are more the results of a set of individual decisions made in haste over a period of time, and then working around the unintended consequences.

Loading the Variables

So, now that we have a little background on the why, we can look at how Magento loads the system configuration variables into the global configuration tree.

Taking a look at our Magento application object’s _initModules method

the call we’re interested in is

The loadDb method is the one that kicks off loading system configuration variables from the database into the global configuration tree. You’ll recall from earlier articles the call to

is where we loaded our module’s config.xml files. So, this means any values hard coded into a config.xmlfile’s <default/>, <websites/>, or <stores/> node are already loaded in the global config.

You may also be interest in the call to

which precedes the loadDb method. This static method call is what kicks of the running of the setup resource scripts (Magento’s version of migrations). We’re not going to go into the how of this, but you should be aware that since we haven’t yet loaded the system configuration variables into the tree, that checking the value of any configuration field from a setup resource script won’t get you the results you want.

Hopping to the config object’s class,

we can the see the loadDb method gets a reference to a resource model object, and then calls it’s loadToXmlmethod, passing the configuration object in as $this

The loadToXml method is where the actual database work of the loading is done. Since the configuration object passes itself in as $this, it’s a safe bet that the actual merging of the configuration will be done in this method as well.

The Configuration Resource Model

So which resource model is it? If we look at the getResourceModel definition, we can see

that the resource model is hard coded, and is instantiated with a getResourceModel method. This resolves to theMage_Core_Model_Resource_Config class, which is located at

app/code/core/Mage/Core/Model/Resource/Config.php

Let’s take a look at the loadToXml method.

Loading the XML

The first thing to pay attention to is the method prototype.

This method accepts only a config object that is, or inherits from, Mage_Core_Model_Config. In our case, this is the global configuration object that was passed in as $this from above. The loadToXml method is going to manipulate this object, and load in any system configuration values it finds in the core_config_data table.

Of course, to do this we’ll need a way to read database values

The _getReadAdapter method is a standard method on every model resource object, and will fetch you a directhandler to the database. This allows you to create SQL queries via a standard Zend style interface. With a read adapter in hand, we’re ready to query the database. The first query is this chunk of code

Here Magento is directly querying the database with SQL that looks something like this

SELECT website_id, code, name FROM core_website 

The core_website table is where Magento persists core/website models, created via the admin console at

System -> Manage Stores

There’s various reasons the core team may be using raw SQL here instead of the model API. As always, the best point of view is to be curious about why something was done, but the accept the pattern even if you disagree with it.

After running the above query, each row is iterated over

and used to set a new node on our configuration object’s global tree. The end result is a tree that looks something like this

with each sub-node of <websites/> representing the information loaded from the database. With this information in the tree, any Magento system or client developer now has access to all the website codes (and their database IDs) loaded in the system.

In addition to creating a permeant record of this information in the global configuration tree, there’s also this line

which stashes the website code into a local array. We’ll be seeing this variable again later, but it’s safe to ignore for now.

Loading the Stores

Next up we see a very similar pattern, but this time it applies to core/store objects (also managed at System -> Manage Stores)

The SQL query that’s run here is

SELECT store_id, code, name, website_id FROM core_store;

Again, rows are iterated over to create a node for each store

You’ll also notice the following line

which dives back into the <websites/> node to set store codes and ids under each website-code node.

This duplication of information is likely a bit of legacy code. While the Magento system developers could easily fix thier own code to only look in the one true place for a piece of information, they can’t go and fix all the code in the world that was written against early versions of Magento.

Back to the code at hand, you’ll also see that we’re stashing this information away in local $stores and$websites variables again

Loading the Persisted Configuration Values

Next up, we’re finally set to load the configuration values from the core_config_data table.

The query we’re constructing above looks something like this

SELECT scope, scope_id, path, value FROM core_config_data;

Also, notice that we’re using the $condition variable to create a WHERE clause. This variable is the loadXmlmethod’s second parameter. In our case it’s not used, and is either legacy or a paramater used internally by the core team during development.

Also, ignore the $substFrom and $substTo variables for now, we’ll get to them eventually.

We now have an array of results in $rowset, with each inner array structured as follows

Each of these inner arrays represent one persisted system configuration variable from the admin console. The rest of the loadXml method is taking this information and loading it into the global configuration tree.

Loading Default Scoped Configuration Variables

Next up is the following

Here Magento is looping through all the rows returned from the core_config_data table. However, notice the conditional guard clause

Magento will skip any iteration of this loop that doesn’t have 'default' as a scope value. That’s because the purpose of this loop is to set only the values for the <default/> node. This is done with a relatively straight forward call to setNode

Here Magento uses the string path stored with each system configuration variable to set a specific node in the global configuration tree. Using x-ray variable vision, that would look something like

The call to setNode will replace any value previously set.

Prior to calling setNode, there’s this curious call

Here Magento is using the $substFrom and $substTo arrays to transform the configuration value after loading it from the database table. What’s curious about this is $substFrom and $substTo are empty arrays, and will always be empty arrays. It’s not clear if this was the start of some configuration substitution system, of if it’s an internal convention used during development and/or testing. Regardless, you can safely ignore this call.

We now have a fully loaded <default/> node, with values from the database superseding values set directly in the configuration. However, we’re not done with the default node yet.

Remember the information we were stashing earlier in the $websites array? Here’s where we’re using it. Magento will iterate over each website id/code pair, and then use it to fetch the entire website node from global config tree. The, our old friend extend is used to copy and merge the contents of the <default/> node into each individual <websites/> node. This means each individual node at websites/[CODE] has a full copy of every configuration value in the system.

Loading Website Scoped Configuration Variables

So, we now have a loaded <default/> configuration node. We also have a <websites/> configuration node loaded with all the values from <default/> (via the extend method). Our next step is to load the values fromcore_config_data with website scope into the <websites/> node.

Here we’re looping through our $rowset again, except this time we’re skipping anything that doesn’t have a website scope

Another difference is we’re checking for the existence of a scope_id key in the $websites array before setting our node

The plain english meaning of this if clause is

Does the website this configuration variable was set for still exist? If so, set the value, if not, stash the website id in the new $deleteWebsites array.

Another slight difference here is the construction of the node path

This code constructs a path in the form

websites/[WEBSITE CODE]/[PERSISTED/VARIABLE/PATH]

When we were setting the <default/> we didn’t need the additional website code node, as there’s only one level of default values.

Loading Store Scoped Configuration Variables

With our <websites/> node fully populated, it’s time to load the values into the <stores/> node. The first step towards this is to copy and merge information from each <websites/> node into a corresponding <stores/>node. That’s done with the following code.

For each id/code pair in $websites, this loop will get a reference to the tree of just set configuration variables

Then, if we previously determined this website had stores

we’ll iterate over each of the stores within that website

and use the node from <websites/> as the source node for the extend copy and merge into thestores/[STORE_CODE] node. Much like <websites/> started with a full copy of <default/>, each sub-code node in <stores/> starts will a full copy of its parent website values.

With this base in place, it’s time to loop through $rowset one more time

This loop is identical to the loop for “websites” scoped values, except that we’re only looking at “stores” scoped values

and setting nodes under the <stores/> node.

and stashing the IDs of stores that don’t exist in $deleteStores

At this point we now have a global configuration tree that has all possible configuration values loaded: Both those configured directly in config.xml, and values persisted to the core_config_data table by the UI. Despite being done, there’s one last bit of housekeeping that needs to happen.

Self Cleaning

While we were inserting values into the <websites/> and <stores/> nodes, we were also populating two arrays

As a reminder (in case you’re as frazzled reading all that as we are writing it), these arrays contain store and website IDs that were encountered in the scope_id column, but whose value didn’t match any website or store object in the system. The reason these values were being stashed is the last bit of code in the loadXml method.

These two blocks of code each construct a DELETE SQL query to remove any configuration records that match the IDs stored in the two arrays.

DELETE FROM core_config_data WHERE scope_id IN (7,8,9);
DELETE FROM core_config_data WHERE scope_id IN (6,4,7);

We’ll be covering rationales for this sort of code in a future article, so for now just be aware it happens, and make sure you never remove a store or website record from the database if there’s important information stored in that store or website’s configuration!

Using the System Configuration Variables

Since we’ve come this far, it’s worth investigating how the system configuration system fetches its values. Right now we’ve loaded values into three top level nodes

but it’s not clear which of these is the “source of truth” for a particular value. That’s where theMage::getStoreConfig method comes into play. It acts as a single point of entry for Magento client developers to retrieve any configuration value by its three part path

This static method is defined on the final Mage class

Let’s break out the method chaining to make this code easier to talk about. The above could be rewritten as

When you call getStoreConfig, Magento gets a reference to an instance of the specific store object you’ve passed in as the second parameter. If this value is omitted (as is usually the case), getStore returns an instance of the current store object. Then, the passed in $path is handed off to the store object’s getConfig method. Let’s take a look at that method

This is where the majority of the work retrieving a configuration value is done. To start, the store object checks it’s local _configCache for a value and returns it

This saves PHP from re-running the code to fetch a config value it’s already retrieved once. Next up,

Here Magento gets a reference to the global configuration tree object with Mage::getConfig. Then, using the passed in configuration path and the store code of the current object, constructs the following path

stores/[CODE HERE]/foo/baz/baz

and then uses that path to retrieve some configuration data via getNode. Next up is this bit of code

If Magento fails to load data at the specified path, AND determines Magento hasn’t been installed yet, it will search for values in the <default/> node. At this point if Magento still hasn’t found a value it gives up, returning null.

So, before we continue, it’s worth noting that despite loading the entire <websites/> node into the configuration object, Magento never consults this node when loading a configuration value. That said, website scope still works as each store is, by definition, part of a website.

Finally, rather than return a value directly, the store object’s getConfig method passes the value into_processConfigValue before returning the requested value to the end user.

The _processConfigValue method is where the raw configuration node is turned into a concrete value for the Magento client programmer. Let’s take a look.

Processing Configuration Values

The _processConfigValue method starts off by also checking the local _configCache for a value.

This is a redundant check in this particular code path, but a little paranoid programming never hurt anyone. Next, if the fetched data/node has any child nodes

then Magento will run though each child, make a single recursive call to _processConfigValue, and store the results in an array. This array is then cached to the local _configCache property, and then the array is returned to the end user. This is what allows you to fetch all the configuration values under a particular node namespace with code like the following

Assuming we’re dealing with a concrete, childless value node, there’s two major tasks the_processConfigValue method needs to accomplish. First, the Magento configuration supports a special attribute on config nodes named backend_model. This attribute allows a developer to programmatically manipulate a configuration value before returning it. The second bit of processing that needs to be done is replacing certain {{template}} {{values}}.

Let’s take a look at both in turn

Magento’s Backend Model Configuration Processing

The processing of the backend_model attribute happens in this code block

First, Magento casts the node as a string to fetch its value. Then, it searches the uncast $node for a backend model attribute. You can see an example of this in the Mage_Usa module’s config.xml.

Remember, the nodes in <default/> are extended into both the <websites/> and <stores/> nodes, so even though the config.xml only has this attribute in the <default/> node, it will carry over to the nodes under <websites/> and <stores/>.

This attribute contains a Magento class alias, which is used to instantiate a model.

Then, both the config value’s path and fetched value are set as data properties of that model, and then the model’safterLoad method is called.

The implicit contract here is the model’s afterLoad method may manipulate the value, which is fetched back out

From what I’ve seen, the only use of this by the core system is to process item values through theadminhtml/system_config_backend_encrypted model’s afterLoad method to ensure user-programmers can fetch the unencrypted value without storing it unencrypted in the database or various config caches.

Template Variable Replacements

Once Magento has processed any backend_model attributes, the last step in processing the configuration value is replacing a set of configuration template variables.

The outer conditional block determines if template replacements need to happen by checking if the string starts with two curly brackets ({{).

The first two conditional leafs of the inner conditional handle {{unsecure_base_url}} and{{secure_base_url}} as a special case. If these are present Magento will fetch another configuration value directly from the config to use in a string substitution at the following paths.

A quick warning: If you set your web/unsecure/base_url value to {{unsecure_base_url}}, you’ll create an endless recursion loop that will kill your entire system. So don’t do that.

The final if clause is a little tricker

This code is actually doing two things. The first is, if the string {{base_url}} is encountered, it will be left alone, unchanged. Secondly, the substDistroServerVars method handles the final substitution of Magento’s other template variables.

Distro Server Vars

The final template variables replaced in the config are

{{root_dir}}, {{app_dir}}, {{var_dir}}, {{base_url}} 

If you take a look at the substDistroServerVars method

you’ll see this list of substitution variables and values it loaded in the getDistroServerVars method. If we look at that method

we see that the values for {{root_dir}}, {{app_dir}}, and {{var_dir}} are loaded from the the options passed into the configuration object at instantiation time, while {{base_url}} comes from transformations performed against PHP’s $_SERVER super global. Of course, as previously discussed, {{base_url}} is skipped in our particular code path, so it’s unclear if this is a bit of legacy code, or if there’s other parts of the core system still relying on this method for other things.

Finally, our value fully processed, it’s stashed in the local _configCache array to ensure further requests for the saved value will be returned immediately.

config configuration